feat(miner-portfolio): add PortfolioQueueManager caps-aware batch claim (#4285) by andriypolanski · Pull Request #4493 · JSONbored/loopover

andriypolanski · 2026-07-09T21:55:05Z

Closes #4285

Summary

Add packages/gittensory-miner/lib/portfolio-queue-manager.js: compose the persisted SQLite store (initPortfolioQueueStore) with the engine's pure nextEligibleItems selector so batch claiming respects globalWipCap / perRepoWipCap and cross-repo diversification.
Extend portfolio-queue.js with a transactional batchClaim(selectFn) hook (BEGIN IMMEDIATE → re-read active rows → claim still-queued targets via UPDATE … RETURNING) so concurrent writers cannot double-claim.
Export pure helpers (entriesToPortfolioQueue, selectEligibleBatch, queueItemId) for unit tests without touching the CLI.

The existing gittensory-miner queue next single-row path is unchanged.

Scope

Caps-aware claimNextBatch() on the manager (constructor/function caps argument only — no .gittensory-miner.yml wiring)
Race-safe batch claim under concurrent writers
In-memory SQLite tests mirroring packages/gittensory-engine/test/portfolio-queue.test.ts scenarios
No UI changes — UI Evidence not required

Test plan

test/unit/miner-portfolio-queue-manager.test.ts — empty queue, per-repo cap saturation, multi-repo diversification, zero caps, concurrent claim safety
npm run build:miner
npm run test:ci green
npm audit --audit-level=moderate clean

Validation

git diff --check
npm run typecheck
npx vitest run test/unit/miner-portfolio-queue-manager.test.ts
npm run build:miner
npm run test:ci
npm audit --audit-level=moderate

Changed files

File	Change
`packages/gittensory-miner/lib/portfolio-queue-manager.js`	New: manager composing store + engine selector
`packages/gittensory-miner/lib/portfolio-queue-manager.d.ts`	Type declarations
`packages/gittensory-miner/lib/portfolio-queue.js`	Add transactional `batchClaim()` for race-safe multi-row claims
`packages/gittensory-miner/lib/portfolio-queue.d.ts`	`batchClaim` on store type
`packages/gittensory-miner/package.json`	Syntax-check new module
`test/unit/miner-portfolio-queue-manager.test.ts`	Unit tests

…im (JSONbored#4285)

superagent-security · 2026-07-09T21:55:46Z

Superagent didn't find any vulnerabilities or security issues in this PR.

loopover-orb · 2026-07-09T21:58:56Z

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

_{Review updated: 2026-07-09 22:10:47 UTC}

6 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

safe to merge

Review summary
This PR adds a caps-aware batch-claim path for the miner's local portfolio queue: a new portfolio-queue-manager.js composes the existing pure engine selector (nextEligibleItems) with a new transactional batchClaim() on the SQLite store, projecting persisted rows into the engine's PortfolioQueueItem shape and mapping selections back to composite (repoFullName, identifier) targets. The transaction correctly uses BEGIN IMMEDIATE plus a per-target status='queued' re-check at claim time, which is genuinely race-safe against concurrent writers on the same file, and the row-order-preserving bucket projection keeps priority ordering intact through the engine's diversification logic. No schema/migration change is needed since it reuses the existing miner_portfolio_queue table, and the .d.ts files match their .js exports exactly.

Nits — 5 non-blocking

packages/gittensory-miner/lib/portfolio-queue-manager.js: closeDefaultPortfolioQueueManager() is a documented no-op stub ('managers are opened explicitly today') — either wire it to a real default-manager singleton mirroring closeDefaultPortfolioQueueStore, or drop the export until something needs it.
test/unit/miner-portfolio-queue-manager.test.ts: the 'does not claim rows another writer already took inside the same transaction window' test calls store.dequeueNext() reentrantly from inside the same selectFn callback on the same connection — it verifies reentrant-callback ordering, not a true separate-connection/process race, so the name overclaims what's actually covered.
packages/gittensory-miner/lib/portfolio-queue-manager.js: parseQueueItemId's '::' separator split assumes repoFullName never contains '::' (true for valid GitHub owner/repo names but not explicitly enforced by normalizeRepoFullName) — harmless in practice but worth a one-line comment noting the assumption.
Consider exporting a small helper or comment clarifying that entriesToPortfolioQueue's bucket-level repoFullName is lowercased (repoKey) while item-level repoFullName keeps original casing — this asymmetry is correct but non-obvious on a first read.
If a future PR wires claimNextBatch() to .gittensory-miner.yml caps, note the current normalizePortfolioCaps default of {globalWipCap:1, perRepoWipCap:1} in initPortfolioQueueManager so config wiring doesn't silently diverge from this default.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#4285
Related work	⚠️ 1 scoped overlap	Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope	❌ 8/20	High review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 136 registered-repo PR(s), 82 merged, 25 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	andriypolanski; Gittensor profile; 136 PR(s), 25 issue(s).
Gate result	✅ Passing	No configured blocker found.

Linked issue satisfaction

Addressed
The PR adds portfolio-queue-manager.js/.d.ts composing initPortfolioQueueStore with the engine's nextEligibleItems via entriesToPortfolioQueue/selectEligibleBatch, adds a race-safe transactional batchClaim to portfolio-queue.js, accepts caps as a plain constructor argument without touching .gittensory-miner.yml, includes in-memory tests covering empty queue, per-repo cap saturation, multi-repo div

Review context

Author: andriypolanski
Role context: outside_contributor
Public audience mode: oss maintainer
Lane context: Repository is configured for direct PR review.
Public profile languages: not available
Official Gittensor activity: 136 PR(s), 25 issue(s).
Related work: Titles/paths share 6 meaningful terms. (issue #4285, issue #4307)

Contributor next steps

Review top overlaps.
Add a concise scope and risk note.
Check active issues and PRs before submitting.

Signal definitions

Related work = same linked issue, overlapping active PRs, or title/path similarity.
Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

Re-run Gittensory review

codecov · 2026-07-09T22:02:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.03%. Comparing base (953fdef) to head (57cbb24).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4493   +/-   ##
=======================================
  Coverage   94.03%   94.03%           
=======================================
  Files         420      420           
  Lines       37521    37521           
  Branches    13701    13701           
=======================================
  Hits        35282    35282           
  Misses       1583     1583           
  Partials      656      656

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

loopover-orb

Gittensory approves — the gate is satisfied and CI is green.

feat(miner-portfolio): add PortfolioQueueManager caps-aware batch cla…

57cbb24

…im (JSONbored#4285)

andriypolanski requested a review from JSONbored as a code owner July 9, 2026 21:55

andriypolanski marked this pull request as draft July 9, 2026 21:55

github-actions Bot deployed to preview/pr-4493 July 9, 2026 21:56 View deployment

loopover-orb Bot added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 9, 2026

andriypolanski marked this pull request as ready for review July 9, 2026 22:02

loopover-orb Bot assigned andriypolanski Jul 9, 2026

loopover-orb Bot approved these changes Jul 9, 2026

View reviewed changes

loopover-orb Bot merged commit 09724bf into JSONbored:main Jul 9, 2026
10 checks passed

loopover-orb Bot removed the gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. label Jul 9, 2026

JSONbored added the gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. label Jul 9, 2026

andriypolanski deleted the feat/portfolio-queue-4285 branch July 16, 2026 15:11

Navigation Menu

Uh oh!

feat(miner-portfolio): add PortfolioQueueManager caps-aware batch claim (#4285) - #4493

feat(miner-portfolio): add PortfolioQueueManager caps-aware batch claim (#4285)#4493
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
andriypolanski:feat/portfolio-queue-4285

andriypolanski commented Jul 9, 2026 •

edited

Loading

Uh oh!

superagent-security Bot commented Jul 9, 2026

Uh oh!

loopover-orb Bot commented Jul 9, 2026 •

edited

Loading

✅ Gittensory review result - approve/merge recommended

Uh oh!

codecov Bot commented Jul 9, 2026

Uh oh!

loopover-orb Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

andriypolanski commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Scope

Test plan

Validation

Changed files

Uh oh!

superagent-security Bot commented Jul 9, 2026

Uh oh!

loopover-orb Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Gittensory review result - approve/merge recommended

Uh oh!

codecov Bot commented Jul 9, 2026

Codecov Report

Uh oh!

loopover-orb Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andriypolanski commented Jul 9, 2026 •

edited

Loading

loopover-orb Bot commented Jul 9, 2026 •

edited

Loading